Provisioning aggregate computational workloads and air conditioning unit configurations to optimize utility of air conditioning units and processing resources within a data center

ABSTRACT

Methods, apparatuses, and computer program products for provisioning aggregate computational workloads and air conditioning unit configurations to optimize utility of air conditioning units and processing resources within a data center are provided. Embodiments include for each air conditioning unit within the data center, determining a thermal zone generated by the air conditioning unit; for a given aggregate computational workload, identifying a plurality of computational workload configurations, each computational workload configuration indicating spatial assignments of the aggregate computational workload among a plurality of processing resources within the data center; for each computational workload configuration, calculating a total minimum energy consumption of the air conditioning units and the processing resources; and selecting the computational workload configuration with the lowest total minimum energy consumption and the determined lowest power air conditioning unit configuration corresponding with the selected computational workload configuration.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of and claims priorityfrom U.S. patent application Ser. No. 13/280,887, filed on Oct. 25,2011.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention is data processing, or, more specifically,methods, apparatuses, and computer program products for provisioningaggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center.

2. Description of Related Art

The development of the EDVAC computer system of 1948 is often cited asthe beginning of the computer era. Since that time, computer systemshave evolved into extremely complicated devices. Today's computers aremuch more sophisticated than early systems such as the EDVAC. Computersystems typically include a combination of hardware and softwarecomponents, application programs, operating systems, processors, buses,memory, input/output devices, and so on. As advances in semiconductorprocessing and computer architecture push the performance of thecomputer higher and higher, more sophisticated computer software hasevolved to take advantage of the higher performance of the hardware,resulting in computer systems today that are much more powerful thanjust a few years ago.

One area that software is being used is in the controlling of energyconsumption of computing systems. In particular, energy consumption hasbecome a critical issue for data centers, triggered by the rise inenergy costs, volatility in the supply and demand of energy and thewidespread proliferation of power-hungry information technology (IT)equipment. In data centers, myriad integrating physical components suchas power distribution units, power supplies, water air conditioningunits, and air conditioning units interact not just with one another,but also with the software components, resulting in a management problemthat is both qualitatively similar to and quantitatively harder thanthat of managing computer hardware alone.

SUMMARY OF THE INVENTION

Methods, apparatuses, and computer program products for provisioningaggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center are provided. Embodimentsinclude for each air conditioning unit within the data center,determining, by a configuration optimizer, a thermal zone generated bythe air conditioning unit; for a given aggregate computational workload,identifying, by the configuration optimizer, a plurality ofcomputational workload configurations, each computational workloadconfiguration indicating spatial assignments of the aggregatecomputational workload among a plurality of processing resources withinthe data center; for each computational workload configuration,calculating, by the configuration optimizer, a total minimum energyconsumption of the air conditioning units and the processing resources,wherein calculating a total minimum energy consumption includesdetermining a lowest power air conditioning unit configuration of theair conditioning units that enables each air conditioning unit toconsume the least amount of energy while maintaining a real-time inlettemperature of the air conditioning unit's thermal zone within apredetermined temperature range; and selecting, by the configurationoptimizer, the computational workload configuration with the lowesttotal minimum energy consumption and the determined lowest power airconditioning unit configuration corresponding with the selectedcomputational workload configuration.

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescriptions of exemplary embodiments of the invention as illustrated inthe accompanying drawings wherein like reference numbers generallyrepresent like parts of exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 sets forth a block diagram of automated computing machinerycomprising an exemplary computer useful in provisioning aggregatecomputational workloads and air conditioning unit configurations tooptimize utility of air conditioning units and processing resourceswithin a data center according to embodiments of the present invention.

FIG. 2 sets forth a diagram of a data center operating according toembodiments of the present invention.

FIG. 3 sets forth a diagram of a data center with thermal zonesdetermined according to embodiments of the present invention.

FIG. 4 sets forth a diagram of a data center with thermal zonesdetermined according to embodiments of the present invention

FIG. 5 sets forth a flow chart illustrating an exemplary method forprovisioning aggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention.

FIG. 6 sets forth a flow chart illustrating a further exemplary methodfor provisioning aggregate computational workloads and air conditioningunit configurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention.

FIG. 7 sets forth a flow chart illustrating a further exemplary methodfor provisioning aggregate computational workloads and air conditioningunit configurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention.

FIG. 8 sets forth a flow chart illustrating a further exemplary methodfor provisioning aggregate computational workloads and air conditioningunit configurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention.

FIG. 9 sets forth a flow chart illustrating a further exemplary methodfor provisioning aggregate computational workloads and air conditioningunit configurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary methods, apparatuses, and computer program products forprovisioning aggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center in accordance with the presentinvention are described with reference to the accompanying drawings,beginning with FIG. 1. Provisioning aggregate computational workloadsand air conditioning unit configurations to optimize utility of airconditioning units and processing resources within a data center inaccordance with the present invention is generally implemented withcomputers, that is, with automated computing machinery. FIG. 1 setsforth a block diagram of automated computing machinery comprising anexemplary computer (152) useful in provisioning aggregate computationalworkloads and air conditioning unit configurations to optimize utilityof air conditioning units and processing resources within a data centeraccording to embodiments of the present invention. The computer (152) ofFIG. 1 includes at least one computer processor (156) or ‘CPU’ as wellas random access memory (168) (‘RAM’) which is connected through a highspeed memory bus (166) and bus adapter (158) to processor (156) and toother components of the computer (152).

Stored in RAM (168) is a configuration optimizer (191) that includescomputer program instructions for provisioning aggregate computationalworkloads and air conditioning unit configurations to optimize utilityof air conditioning units and processing resources within a data center.The configuration optimizer (191) includes computer program instructionsthat when executed by the processor (156) cause the computer (152) tocarry out the step of: for each air conditioning unit within the datacenter, determining, by the configuration optimizer (191), a thermalzone generated by the air conditioning unit. An air conditioning unitmay include a computer room air-conditioning unit (CRAC). A thermal zoneis a region of a data center that a particular air conditioning unitcools.

The configuration optimizer (191) also includes computer programinstructions that when executed by the processor (156) cause thecomputer (152) to carry out the step of identifying, for a givenaggregate computational workload by the configuration optimizer (191), aplurality of computational workload configurations. Identifyingcomputational workload configurations may include identifying one ormore known configurations, determining all possible configurations, orany variation thereof as will occur to one of skill in the art. Eachcomputational workload configuration indicates spatial assignments ofthe aggregate computational workload among a plurality of processingresources within the data center.

The configuration optimizer (191) includes computer program instructionsthat when executed by the processor (156) cause the computer (152) tocarry out the step of calculating, for each computational workloadconfiguration by the configuration optimizer (191), a total minimumenergy consumption of the air conditioning units and the processingresources. A total minimum energy consumption of the air conditioningunits and the processing resources is a value that indicates how muchenergy is consumed by the air conditioning units and the processingresources when the air conditioning units are operating at their mostefficient configuration at a particular workload configuration of theprocessing resources. The air conditioning units can have different airconditioning unit configurations by turning on one or more airconditioning units; turning off one or more air conditioning units;changing the temperature of the airflow of one or more air conditioningunits; and changing the fan speed of one or more of the air conditioningunits. Calculating a total minimum energy consumption includesdetermining a lowest power air conditioning unit configuration of theair conditioning units that enables each air conditioning unit toconsume the least amount of energy while maintaining a real-time inlettemperature of the air conditioning unit's thermal zone within apredetermined temperature range. A lowest power air conditioning unitconfiguration is the most efficient configuration of the airconditioning units. The predetermined temperature range may be a safeoperating temperature range for the electrical components in the datacenter.

The configuration optimizer (191) also includes computer programinstructions that when executed by the processor (156) cause thecomputer (152) to carry out the step of selecting, by the configurationoptimizer (191), the computational workload configuration with thelowest total minimum energy consumption and the determined lowest powerair conditioning unit configuration corresponding with the selectedcomputational workload configuration. That is, the configurationoptimizer (191) is configured to determine the optimal computationalworkload configuration and air conditioning unit configuration thatmaximizes utility of the data center while operating within a particulartemperature range.

Also stored in RAM (168) is an operating system (154). Operating systemsuseful for provisioning aggregate computational workloads and airconditioning unit configurations to optimize utility of air conditioningunits and processing resources within a data center according toembodiments of the present invention include UNIX™ Linux™ Microsoft XP™AIX™ IBM's i5/OS™ and others as will occur to those of skill in the art.The operating system (154) and the configuration optimizer (191) in theexample of FIG. 1 are shown in RAM (168), but many components of suchsoftware typically are stored in non-volatile memory also, such as, forexample, on a disk drive (170).

The computer (152) of FIG. 1 includes disk drive adapter (172) coupledthrough expansion bus (160) and bus adapter (158) to processor (156) andother components of the computer (152). Disk drive adapter (172)connects non-volatile data storage to the computer (152) in the form ofdisk drive (170). Disk drive adapters useful in computers forprovisioning aggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention include Integrated Drive Electronics (‘IDE’)adapters, Small Computer System Interface (‘SCSI’) adapters, and othersas will occur to those of skill in the art. Non-volatile computer memoryalso may be implemented for as an optical disk drive, electricallyerasable programmable read-only memory (so-called ‘EEPROM’ or ‘Flash’memory), RAM drives, and so on, as will occur to those of skill in theart.

The example computer (152) of FIG. 1 includes one or more input/output(‘I/O’) adapters (178). I/O adapters implement user-orientedinput/output through, for example, software drivers and computerhardware for controlling output to display devices such as computerdisplay screens, as well as user input from user input devices (181)such as keyboards and mice. The example computer (152) of FIG. 1includes a video adapter (183), which is an example of an I/O adapterspecially designed for graphic output to a display device (180) such asa display screen or computer monitor. Video adapter (183) is connectedto processor (156) through a high speed video bus (164), bus adapter(158), and the front side bus (162), which is also a high speed bus.

The exemplary computer (152) of FIG. 1 includes a communications adapter(167) for data communications with other computers (182) and for datacommunications with a data communications network (100). Such datacommunications may be carried out serially through RS-232 connections,through external buses such as a Universal Serial Bus (‘USB’), throughdata communications networks such as IP data communications networks,and in other ways as will occur to those of skill in the art.Communications adapters implement the hardware level of datacommunications through which one computer sends data communications toanother computer, directly or through a data communications network.Examples of communications adapters useful for provisioning aggregatecomputational workloads and air conditioning unit configurations tooptimize utility of air conditioning units and processing resourceswithin a data center according to embodiments of the present inventioninclude modems for wired dial-up communications, Ethernet (IEEE 802.3)adapters for wired data communications network communications, and802.11 adapters for wireless data communications network communications.

FIG. 2 sets forth a diagram of a data center (200) operating accordingto embodiments of the present invention. The data center (200) of FIG. 2includes processing resources. Processing resources may includeprocessing racks, power distribution units (PDU), and other types ofenergy consuming electrical devices commonly found in data centers forthe processing of data. A processing rack is a configuration ofelectrical components, such as computer processors, data storagedevices, servers, and power supplies. In the example of FIG. 2, the datacenter (200) includes processing racks (204) that receive power from apower distribution unit (PDU) (208). The data center (200) of FIG. 2also includes an air conditioning unit (202) to generate an outputairflow (230) that is directed into a plenum (280) beneath the raisedfloor (210) of the data center. In addition, the data center of FIG. 2may also include an additional air conditioning unit (not shown) thatcan provide an output air flow (292) into the plenum (280) and receive areturn airflow (290). The output airflow (230) flows from the plenum andthrough perforated tiles (206) into the area above the raised floor(210). The data center (200) of FIG. 2 also includes perforated ceilingtiles to allow return airflow (232) to travel back into the airconditioning unit (202). In this manner, the air conditioning unit (202)acts to cool the processing resources that generate heat.

Attached to the first air conditioning unit (202) is an air conditioningunit sensor (212) that is configured to measure the pressure of theoutput airflow (230) generated by the air conditioning unit (202). Inaddition, perforated tile sensors (218) are attached to the perforatedtiles (206) in the data center (200) to measure the pressure drop acrossa particular perforated tile (206). The generated pressure measured bythe air conditioning unit sensor (212) and the pressure drop measured bythe perforated tiles (206) may be transmitted to the configurationoptimizer (191) of FIG. 1.

The configuration optimizer (191) of FIG. 1 may use the generatedpressure and the pressure drop to determine the thermal zones of thedata center (200). A thermal zone is a region of a data center that aparticular air conditioning unit cools. A region of the data center mayindicate a two dimensional area of the data center floor or a threedimensional space that includes the space above the area of the datacenter floor. To determine a thermal zone, the configuration optimizer(191) of FIG. 1 may be configured to use the pressure drop informationto determine which perforated tiles in the data center (202) arereceiving the output airflow (230) from the air conditioning unit (202).All of the determined perforated tiles in the data center (202) that arereceiving the output airflow from the air conditioning unit (202) areassigned by the configuration optimizer to the thermal zone of the firstair conditioning unit (202). Although the data center (200) of FIG. 2 isillustrated with only a single air conditioning unit, the airconditioning unit (202), a data center may have more than one airconditioning unit and a configuration optimizer may be configured todetermine multiple thermal zones, where each thermal zone corresponds toa particular air conditioning unit.

FIG. 3 sets forth a diagram of a data center (300) with thermal zonesdetermined according to embodiments of the present invention. The datacenter (300) of FIG. 3 includes a first air conditioning unit (302), asecond air conditioning unit (304), and a third air conditioning unit(306). A configuration optimizer, such as the configuration optimizer(191) of FIG. 1, may use pressure measurements from air conditioningunit sensors and perforated tile sensors to determine the thermal zonesof each air conditioning unit. For example, when, according to the airconditioning unit sensors, the first air conditioning unit (302) isgenerating an output airflow and the second air conditioning unit (304)and third air conditioning unit (306) are not, the configurationoptimizer may assign to a first thermal zone (310), each perforated tilethat has a corresponding perforated tile sensor indicating a pressuredrop. Likewise, a configuration optimizer may use the pressure dropinformation received from the air conditioning unit sensors andperforated tile sensors to determine which perforated tiles are within asecond thermal zone (312) and a third thermal zone (314). Although thethermal zones in the example of FIG. 3 are represented by a particularcontour, the thermal zones may include round edges or have other shapesand dimensions as will occur to one of skill in the art.

As part of the process for determining which regions of the data centercorrespond with a particular thermal zone, the configuration optimizermay be configured to calculate a velocity field indicating thetrajectory of airflow generated by the air conditioning unit. Tocalculate the velocity field of the airflow, the configuration optimizermay utilize equations and principles of fluid dynamics. For example, theconfiguration optimizer may determine the trajectory of the airflow as afunction of potential flow theory. Potential flow theory describes thevelocity field as the gradient of a scalar function: the velocitypotential.

To determine the velocity field under potential flow theory, theconfiguration optimizer may utilize the Poisson equation which isdefined as:

Δφ=f

In the Poisson equation, (Δ) is the Laplace operator, and f and φ arereal or complex-valued functions on a manifold. When the manifold is anEuclidean space, the Laplace operator is often denoted as ∇² and soPoisson's equation is frequently written as:

∇² φ=f

where f is the source or sink of the potential. The gradient of thepotential gives the velocity field v:

v=∇φ

In this model, air conditioning units are treated as sources whileperforated tiles in the raised floor of the data center are treated assinks. The strengths of the sources are obtained in real-time from airconditioning unit sensors mounted on every air conditioning unit. Thestrengths of the sinks are calculated by correlating the pressure dropacross a perforated tile to the flow resistance offered by theperforated tile. The pressure drop across a perforated tile is obtainedin real-time by perforated tile sensors placed in the plenum. That is,the configuration optimizer may use the measured strength of the airconditioning unit along with the recorded pressure drop within theplenum to generate a velocity field that indicates the trajectory ofairflow generated by a particular air conditioning unit.

The configuration optimizer may further refine the velocity field of theairflow based on a model of how airflow is affected by the area of theparticular plenum. That is, the configuration optimizer may determinehow the airflow propagates and reflects off of the surfaces of theparticular plenum and the objects within the plenum. For example, theconfiguration optimizer may utilize a ray tracing approach to determinehow the particles within the airflow reflect off of electricalcomponents, cords, wires within the plenum as well as the generalsurfaces of the plenum. Ray tracing is a method for calculating the pathof waves or particles through a system with regions of varyingpropagation velocity, absorption characteristics, and reflectingsurfaces. In this example, ray tracing indicates how particulars of theairflow change direction or reflect off surfaces within the plenum. Raytracing solves the problem by repeatedly advancing idealized narrowbeams called rays through the medium by discrete amounts. Simpleproblems can be analyzed by propagating a few rays using simplemathematics. More detailed analyses can be performed by using a computerto propagate many rays.

Once a velocity field indicating the trajectory of airflow for a desiredair conditioning unit is obtained, thermal zones for a particular airconditioning unit setting are obtained by an algorithm which calculatesthe trajectory of the air flow using:

x=∫vdt

The configuration optimizer calculates a trajectory of the airflow untilit either intersects with a cooling unit or with a previously assignedpoint. The configuration optimizer uses the calculated trajectory of theairflow to determine which areas of the data center receive airflow froma particular air conditioning unit. The configuration optimizer assignsthe determined area to a particular thermal zone, which is then used forprovisioning aggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center according to embodiments ofthe present invention.

FIG. 4 sets forth a diagram of a data center (300) with thermal zonesdetermined according to embodiments of the present invention. In theexample of FIG. 4, the first thermal zone (310), the second thermal zone(312), and the third thermal zone (314) of the data center (300) of FIG.3 are illustrated. For simplicity, the first air conditioning unit(302), the second air conditioning unit (304), and the third airconditioning unit (306) are not illustrated. However, in the example ofFIG. 4, the processing resources of the data center (300) areillustrated.

As part of the process of using thermal zones for provisioning aggregatecomputational workloads and air conditioning unit configurations tooptimize utility of air conditioning units and processing resourceswithin a data center, a configuration optimizer may be configured todetermine the processing resources of the different thermal zones. Forexample, the configuration optimizer may determine that the firstcomputer processing racks (402) and a first power distribution unit(404) are near perforated tiles within the first thermal zone (310). Theconfiguration optimizer may also assign the second computer processingracks (406) and second PDU (408) to the second thermal zone (312) andassign the third processing racks (410) and a third PDU (412) to thethird thermal zone (314).

The configuration optimizer may also be configured to determine theamount of heat dissipated in the raised-floor area in a particularthermal zone. For example, the configuration optimizer may be configuredto map each computer processing rack to its nearest PDU. Given thismapping, the configuration optimizer may assume that all computerprocessing racks are identical and may estimate the power dissipated bya given rack by determining the total power distributed by a PDU and thenumber of computer processing racks mapped to the PDU. The total powerdissipated in a thermal zone above raised floor (=P^(d) _(RF)), can befound by adding the power dissipated in the processing racks (=P^(d)_(IT)), air conditioning units (=P^(d) _(CRAC)), PDUs (=P^(d) _(PDU)),and other assets (=P^(d) _(Misc)) which lie in that zone:

P ^(d) _(RF) =P ^(d) _(CRAC) +P ^(d) _(PDU) +P ^(d) _(IT)

The cooling power of an air conditioning unit for a thermal zone iscalculated from the fluid volumetric-flow rate (φ), air density (ρ),specific heat of the air (C_(p)), and the difference between the airinlet and the exit temperatures of the thermal zone (ΔT):

P _(Cool) =ΣP _(i,Cool)=Σρφ_(i) C _(p) ΔT

The configuration optimizer performs summation over all the perforatedtiles in that thermal zone. Under steady state conditions, the totalpower dissipated in the thermal zone above the raised floor (=P^(d)_(RF)) should be equal to the total cooling power of the airconditioning unit, P_(Cool). The relationship between the cooling powerof the air conditioning units and the power dissipated by the processingresources may be defined by an energy balance equation such as:

P _(Cool) =P ^(d) _(RF)

The configuration optimizer uses the energy balance equation todetermine the temperature difference (ΔT) across a particular thermalzone. The inlet temperature to the thermal zone is obtained using an airconditioning unit sensor mounted on a corresponding air conditioningunit. Using the energy balance equation above and the inlet temperature,the configuration optimizer may calculate the return temperature to theair conditioning unit for the thermal zone. The configuration optimizermay use the calculated return temperature as part of the determinationof whether the real-time inlet temperature of a thermal zone is within apredetermined temperature range. As part of the process for determiningoptimized air conditioning unit configurations, the configurationoptimizer uses the determination of whether a particular airconditioning unit configuration keeps all of the thermal zones withinthe predetermined temperature range.

For further explanation, FIG. 5 sets forth a flow chart illustrating anexemplary method for provisioning aggregate computational workloads andair conditioning unit configurations to optimize utility of airconditioning units and processing resources within a data centeraccording to embodiments of the present invention. The method of FIG. 5includes for each air conditioning unit within the data center,determining (502), by a configuration optimizer (191), a thermal zonegenerated by the air conditioning unit. Determining (502) a thermal zonegenerated by the air conditioning unit may be carried out by calculatingthe strength of an air conditioning unit supplying an output airflowinto a data center plenum; calculating pressure drops between theperforated tiles and the plenum; based on the strength of the airconditioning unit and the pressure drops across the perforated tiles,determining a trajectory of the airflow in the plenum; and based on thetrajectory of the airflow, identifying the areas of the data center thatare cooled by the particular air conditioning unit.

The method of FIG. 5 includes for a given aggregate computationalworkload, identifying (504), by the configuration optimizer (191), aplurality (530) of computational workload configurations. Eachcomputational workload configuration indicates assignments of theaggregate computational workload among a plurality of processingresources within the data center. Assigning an aggregate computationalworkload among a plurality of processing resources, or ‘load balancing’as it is sometimes called, is a computer networking methodology todistribute computational workloads across multiple computers or acomputer cluster, network links, central processing units, disk drives,or other resources, to achieve optimal resource utilization, maximizethroughput, minimize response time, and avoid overload. Using multiplecomponents with load balancing, instead of a single component, mayincrease reliability through redundancy. Identifying (504) a plurality(530) of computational workload configurations may be carried out bydetermining for each processing resource, the available processingcapacity of each processing resource as a percentage of the totalprocessing capacity of the processing resource; determining whichprocesses of the processing resources can be moved to other processingresources; and based on the available processing capacities of theprocessing resources, determining different configurations of theprocessing resources assigned with different computational workloads.

The method of FIG. 5 includes for each computational workloadconfiguration, calculating (506), by the configuration optimizer (191),a total minimum energy consumption (532) of the air conditioning unitsand the processing resources. Calculating (506) a total minimum energyconsumption (532) of the air conditioning units and the processingresources may be carried out by determining the energy consumption ofthe processing resources at a particular computational workloadconfiguration; and determining the energy consumption of the airconditioning units at different configurations.

In the method of FIG. 5, calculating (506) a total minimum energyconsumption (532) includes determining (508) a lowest power airconditioning unit configuration (536) of the air conditioning units thatenables each air conditioning unit to consume the least amount of energywhile maintaining a real-time inlet temperature (540) of the airconditioning unit's thermal zone within a predetermined temperaturerange (560). A real-time inlet temperature may be a server inlettemperature of a server on the floor of the data center; a temperatureat a floor sensor; or any other temperature at a location within thedata center. The predetermined temperature range (560) may correspondwith the safe operating limits published by The American Society ofHeating, Refrigerating and Air-Conditioning Engineers (ASHRAE). Forexample, a safe operating range for all IT equipment may be 65 degreesto 80 degrees Fahrenheit (18.3 degrees to 26.6 degrees Celsius).Determining a lowest power air conditioning unit configuration (536) ofthe air conditioning units that enables each air conditioning unit toconsume the least amount of energy while maintaining a real-time inlettemperature (540) of the air conditioning unit's thermal zone within apredetermined temperature range (560) may be carried out by determiningthe real-time inlet temperature of each thermal zone when the airconditioning units are in a particular configuration and when theprocessing resources are operating at the computational workloadconfiguration; and determining which air conditioning unit configurationrequires the lowest amount of energy to operate while maintaining thetemperature of the thermal zone within the predetermined range. Changingfrom one air conditioning unit configuration to another may be carriedout by turning on one or more air conditioning unit; turning off one ormore air conditioning unit; changing the temperature of the airflow ofone or more air conditioning units; and changing the fan speed of one ormore of the air conditioning units.

The method of FIG. 5 includes selecting (510), by the configurationoptimizer (191), the computational workload configuration (538) with thelowest total minimum energy consumption and the determined lowest powerair conditioning unit configuration corresponding with the selectedcomputational workload configuration (538). Selecting (510) thecomputational workload configuration (538) with the lowest total minimumenergy consumption and the determined lowest power air conditioning unitconfiguration corresponding with the selected computational workloadconfiguration (538) may be carried out by storing, for eachcomputational workload configuration, a value representing the energyconsumed by the processing resources and the air conditioning units atthe lowest power air conditioning unit configuration; comparing thestored energy values; and selecting the computational workloadconfiguration with the corresponding lowest stored energy value.

For further explanation, FIG. 6 sets forth a flow chart illustrating afurther exemplary method for provisioning aggregate computationalworkloads and air conditioning unit configurations to optimize utilityof air conditioning units and processing resources within a data centeraccording to embodiments of the present invention. The method of FIG. 6is similar to the method of FIG. 5 in that the method of FIG. 6 alsoincludes for each air conditioning unit within the data center,determining (502) a thermal zone generated by the air conditioning unit;for a given aggregate computational workload, identifying (504), by theconfiguration optimizer (191), a plurality (530) of computationalworkload configurations; for each computational workload configuration,calculating (506) a total minimum energy consumption (532) of the airconditioning units and the processing resources; determining (508) alowest power air conditioning unit configuration (536) of the airconditioning units that enables each air conditioning unit to consumethe least amount of energy while maintaining a real-time inlettemperature (540) of the air conditioning unit's thermal zone within apredetermined temperature range (560); and selecting (510) thecomputational workload configuration (538) with the lowest total minimumenergy consumption and the determined lowest power air conditioning unitconfiguration corresponding with the selected computational workloadconfiguration (538).

The method of FIG. 6, however, includes assigning (602), by theconfiguration optimizer (191), the aggregate computational workload tothe plurality of processing resources in accordance with the selectedcomputational workload configuration (538). Assigning (602) theaggregate computational workload to the plurality of processingresources in accordance with the selected computational workloadconfiguration (538) may be carried out by moving one or more processesto one or more processing resources in accordance with the selectedcomputational workload configuration.

The method of FIG. 6 also includes operating (604), by the configurationoptimizer (191), the air conditioning units in accordance with thedetermined lowest power air conditioning unit configuration (536)corresponding with the selected computational workload configuration(538). Operating (604) the air conditioning units in accordance with thedetermined lowest power air conditioning unit configuration (536)corresponding with the selected computational workload configuration(538) may be carried out by turning on one or more air conditioningunits; turning off one or more air conditioning units; changing thetemperature of the airflow of one or more air conditioning units; andchanging the fan speed of one or more of the air conditioning units.

For further explanation, FIG. 7 sets forth a flow chart illustrating afurther exemplary method for provisioning aggregate computationalworkloads and air conditioning unit configurations to optimize utilityof air conditioning units and processing resources within a data centeraccording to embodiments of the present invention. The method of FIG. 7is similar to the method of FIG. 5 in that the method of FIG. 7 alsoincludes for each air conditioning unit within the data center,determining (502) a thermal zone generated by the air conditioning unit;for a given aggregate computational workload, identifying (504), by theconfiguration optimizer (191), a plurality (530) of computationalworkload configurations; for each computational workload configuration,calculating (506) a total minimum energy consumption (532) of the airconditioning units and the processing resources; determining (508) alowest power air conditioning unit configuration (536) of the airconditioning units that enables each air conditioning unit to consumethe least amount of energy while maintaining a real-time temperature(540) of the air conditioning unit's thermal zone within a predeterminedtemperature range (560); and selecting (510) the computational workloadconfiguration (538) with the lowest total minimum energy consumption andthe determined lowest power air conditioning unit configurationcorresponding with the selected computational workload configuration(538).

In the method of FIG. 7, determining (508) a lowest power airconditioning unit configuration (536) of the air conditioning unitsincludes receiving (702) from an air conditioning unit sensor, anindication (730) of the pressure generated by the air conditioning unit.Receiving (702) from an air conditioning unit sensor, an indication(730) of the pressure generated by the air conditioning unit may becarried out by measuring by the air conditioning unit sensor thepressure generated by the air conditioning unit; transmitting by the airconditioning unit sensor the measured pressure to the configurationoptimizer; and storing by the configuration optimizer the receivedmeasured pressure.

In the method of FIG. 7, determining (508) a lowest power airconditioning unit configuration (536) of the air conditioning unitsincludes receiving (704) from a plurality of perforated tile sensors,indications (732) of pressure drop across each perforated tile.Receiving (704) from a plurality of perforated tile sensors, indications(732) of pressure drop across each perforated tile may be carried out bymeasuring by the perforated tile sensor the pressure drop across aperforated tile; transmitting by the perforated tile sensor the measuredpressure drop to the configuration optimizer; and storing by theconfiguration optimizer, the measured pressure drop.

In the method of FIG. 7, determining (508) a lowest power airconditioning unit configuration (536) of the air conditioning unitsincludes determining (706) a region (734) cooled by the air conditioningunit based on the indication (730) of the generated pressure and theindications (732) of the pressure drop across the perforated tiles.Determining (706) a region (734) cooled by the air conditioning unitbased on the indication (730) of the generated pressure and theindications (732) of the pressure drop across the perforated tiles maybe carried out by: corresponding an indication of a pressure drop with asection of the raised floor; corresponding a region above the floor withthe thermal zone; and grouping the corresponding sections of the raisedfloor of the data center into an area or a region of the data center toform a thermal zone.

In the method of FIG. 7, determining a lowest power air conditioningunit configuration (536) of the air conditioning units includesassigning (708) the determined region (734) to a thermal zone. Assigning(708) the determined region (734) to a thermal zone may be carried outby corresponding section of the area of the data center with aparticular thermal zone; and storing the assignments in a thermal zonemap.

For further explanation, FIG. 8 sets forth a flow chart illustrating afurther exemplary method for provisioning aggregate computationalworkloads and air conditioning unit configurations to optimize utilityof air conditioning units and processing resources within a data centeraccording to embodiments of the present invention. The method of FIG. 8is similar to the method of FIG. 5 in that the method of FIG. 8 alsoincludes for each air conditioning unit within the data center,determining (502) a thermal zone generated by the air conditioning unit;for a given aggregate computational workload, identifying (504), by theconfiguration optimizer (191), a plurality (530) of computationalworkload configurations; for each computational workload configuration,calculating (506) a total minimum energy consumption (532) of the airconditioning units and the processing resources; determining a lowestpower air conditioning unit configuration (536) of the air conditioningunits that enables each air conditioning unit to consume the leastamount of energy while maintaining a real-time inlet temperature (540)of the air conditioning unit's thermal zone within a predeterminedtemperature range (560); and selecting (510) the computational workloadconfiguration (538) with the lowest total minimum energy consumption andthe determined lowest power air conditioning unit configurationcorresponding with the selected computational workload configuration(538).

In the method of FIG. 8, determining a lowest power air conditioningunit configuration (536) of the air conditioning units includesidentifying (802) the processing resources within the thermal zone.Identifying (802) the processing resources within the thermal zone maybe carried out by determining the area of the thermal zone; andcorresponding locations of the processing resources with the areas inthe thermal zones.

In the method of FIG. 8, determining a lowest power air conditioningunit configuration (536) of the air conditioning units includescalculating (804) the power dissipated (832) by the processing resourceswithin the thermal zone. Calculating (804) the power dissipated (832) bythe processing resources within the thermal zone may be carried out bydetermining the heat generated by each processing resource; and summingthe heat generated by each processing resource determined to be withinthe thermal zone.

In the method of FIG. 8, determining a lowest power air conditioningunit configuration (536) of the air conditioning units includescalculating (806) the cooling power (836) of the air conditioning unitassociated with the thermal zone. The cooling power (836) of the airconditioning unit is a cooling capacity of an air conditioning unit at agiven configuration indicating fan speed, temperature and power suppliedto an air conditioning unit. That is, the cooling power is a heat loadthat can be removed by the air conditioning unit at the giveconfiguration. Calculating (806) the cooling power (836) of the airconditioning unit associated with the thermal zone may be carried out bydetermining the volumetric-flow rate of the air conditioning unit;determining the air density of the area about the raised floor of thedata center; determining the specific heat of the air; determining thedifference between the air inlet and the exit temperatures to thethermal zones; and multiplying each of the determined values.

In the method of FIG. 8, determining a lowest power air conditioningunit configuration (536) of the air conditioning units includes based onthe calculated dissipated power (832) and the calculated cooling power(836), determining (808) a real-time inlet temperature (540) of thethermal zone. Determining (808) a real-time inlet temperature (540) ofthe thermal zone may be carried out by measuring with air conditioningunit sensors a real time inlet temperature; and using an energybalancing formula to determine the outlet temperature.

For further explanation, FIG. 9 sets forth a flow chart illustrating afurther exemplary method for provisioning aggregate computationalworkloads and air conditioning unit configurations to optimize utilityof air conditioning units and processing resources within a data centeraccording to embodiments of the present invention. The method of FIG. 9is similar to the method of FIG. 5 in that the method of FIG. 9 alsoincludes for each air conditioning unit within the data center,determining (502) a thermal zone generated by the air conditioning unit;for a given aggregate computational workload, identifying (504), by theconfiguration optimizer (191), a plurality (530) of computationalworkload configurations; for each computational workload configuration,calculating (506) a total minimum energy consumption (532) of the airconditioning units and the processing resources; determining a lowestpower air conditioning unit configuration (536) of the air conditioningunits that enables each air conditioning unit to consume the leastamount of energy while maintaining a real-time inlet temperature (540)of the air conditioning unit's thermal zone within a predeterminedtemperature range (560); and selecting (510) the computational workloadconfiguration (538) with the lowest total minimum energy consumption andthe determined lowest power air conditioning unit configurationcorresponding with the selected computational workload configuration(538).

The method of FIG. 9 is also similar to the method of FIG. 8 in that themethod of FIG. 9 also includes: identifying (802) the processingresources within the thermal zone; calculating (804) the powerdissipated (832) by the processing resources within the thermal zone;calculating (806) the cooling power (836) of the air conditioning unitassociated with the thermal zone; and based on the calculated dissipatedpower (832) and the calculated cooling power (836), determining (808) areal-time inlet temperature (540) of the thermal zone.

In the method of FIG. 9, calculating (804) the power dissipated (832) bythe processing resources within the thermal zone includes mapping (902)a computer processing rack to a nearest power distribution unit (PDU).Mapping (902) a computer processing rack to a nearest power distributionunit (PDU) may be carried out by determining the distance between eachprocessing rack and PDUs within the data center; comparing the measureddistances; and associating each computer processing rack with the PDUthat is closest.

In the method of FIG. 9, calculating (804) the power dissipated (832) bythe processing resources within the thermal zone includes determining(904) the power distributed by the PDU to the computer processing rack.Determining (904) the power distributed by the PDU to the computerprocessing rack may be carried out by measuring the power supplied tothe PDU; receiving an indication of the measured power supplied to thePDU; and receiving an indication of power distributed by the PDU to eachof the processing racks.

In the method of FIG. 9, calculating (804) the power dissipated (832) bythe processing resources within the thermal zone includes calculating(906) the power dissipated by the computer processing rack based on thedetermined power distributed by the PDU to the computer processing rack.Calculating (906) the power dissipated by the computer processing rackbased on the determined power distributed by the PDU to the computerprocessing rack may be carried out by determining the heat generationspecifications of the computer processing rack; and determining the heatgenerated by the computer processing rack in accordance with the heatgeneration specifications and the amount of power supplied by the PDU tothe particular computer processing rack.

Exemplary embodiments of the present invention are described largely inthe context of a fully functional computer system for provisioningaggregate computational workloads and air conditioning unitconfigurations to optimize utility of air conditioning units andprocessing resources within a data center. Readers of skill in the artwill recognize, however, that the present invention also may be embodiedin a computer program product disposed upon computer readable storagemedia for use with any suitable data processing system. Such computerreadable storage media may be any storage medium for machine-readableinformation, including magnetic media, optical media, or other suitablemedia. Examples of such media include magnetic disks in hard drives ordiskettes, compact disks for optical drives, magnetic tape, and othersas will occur to those of skill in the art. Persons skilled in the artwill immediately recognize that any computer system having suitableprogramming means will be capable of executing the steps of the methodof the invention as embodied in a computer program product. Personsskilled in the art will recognize also that, although some of theexemplary embodiments described in this specification are oriented tosoftware installed and executing on computer hardware, nevertheless,alternative embodiments implemented as firmware or as hardware are wellwithin the scope of the present invention.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

It will be understood from the foregoing description that modificationsand changes may be made in various embodiments of the present inventionwithout departing from its true spirit. The descriptions in thisspecification are for purposes of illustration only and are not to beconstrued in a limiting sense. The scope of the present invention islimited only by the language of the following claims.

1. A method of provisioning aggregate computational workloads and airconditioning unit configurations to optimize utility of air conditioningunits and processing resources within a data center, the methodcomprising: for each air conditioning unit within the data center,determining, by a configuration optimizer, a thermal zone generated bythe air conditioning unit; for a given aggregate computational workload,identifying, by the configuration optimizer, a plurality ofcomputational workload configurations, each computational workloadconfiguration indicating spatial assignments of the aggregatecomputational workload among a plurality of processing resources withinthe data center; for each computational workload configuration,calculating, by the configuration optimizer, a total minimum energyconsumption of the air conditioning units and the processing resources,wherein calculating a total minimum energy consumption includesdetermining a lowest power air conditioning unit configuration of theair conditioning units that enables each air conditioning unit toconsume the least amount of energy while maintaining a real-time inlettemperature of the air conditioning unit's thermal zone within apredetermined temperature range; and selecting, by the configurationoptimizer, the computational workload configuration with the lowesttotal minimum energy consumption and the determined lowest power airconditioning unit configuration corresponding with the selectedcomputational workload configuration.
 2. The method of claim 1 furthercomprising: assigning, by the configuration optimizer, the givenaggregate computational workload to the plurality of processingresources in accordance with the selected computational workloadconfiguration; and operating, by the configuration optimizer, the airconditioning units in accordance with the determined lowest power airconditioning unit configuration corresponding with the selectedcomputational workload configuration.
 3. The method of claim 1 whereindetermining a thermal zone generated by the air conditioning unitincludes: receiving from an air conditioning unit sensor, an indicationof the pressure generated by the air conditioning unit; receiving from aplurality of perforated tile sensors, indications of pressure dropacross each perforated tile; determining a region within the data centercooled by the air conditioning unit based on the indication of thegenerated pressure and the indications of the pressure drop across theperforated tiles; and assigning the determined region to a thermal zone.4. The method of claim 1 wherein determining a lowest power airconditioning unit configuration of the air conditioning units thatenables each air conditioning unit to consume the least amount of energywhile maintaining the real-time inlet temperature of the airconditioning unit's thermal zone within a predetermined temperaturerange includes: for each thermal zone: identifying the processingresources within the thermal zone; calculating the power dissipated bythe processing resources within the thermal zone; calculating thecooling power of the air conditioning unit associated with the thermalzone; and based on the calculated dissipated power and the calculatedcooling power, determining a real-time inlet temperature of the thermalzone.
 5. The method of claim 4 wherein calculating the power dissipatedby the processing resources within the thermal zone includes: mapping acomputer processing rack to a nearest power distribution unit (PDU);determining the power distributed by the PDU to the computer processingrack; and calculating the power dissipated by the computer processingrack based on the determined power distributed by the PDU to thecomputer processing rack;
 6. The method of claim 1 wherein thepredetermined temperature range indicates the temperatures within whichthe processing resources of the data center are recommended to operate.7. The method of claim 1 wherein the plurality of processing resourcesinclude at least one of power distribution units, power supplies, andcomputer processing racks. 8-20. (canceled)